Demands on computer systems to perform greater numbers of complex and critical functions has led to the distribution of functions, or portions thereof, to specialized systems. These specialized systems can then be optimized for specific tasks to increase efficiency at the expense of being loosely coupled, which increases the complexities involved with system integration.
For some specialized systems, the complexity of full integration can be infeasible and has been avoided by maintaining redundant data, and applications to process the data. In an engineering company, for example, significant differences exist between systems for accounting and engineering groups. The accounting systems maintain data related to budget, man-hours, pay scales, and taxes, while the engineering systems maintain data related to engineering analyses, materials, designs, man-hours and budget. Although access to data regarding man-hours, budgets, and budgetary projections for the engineering groups by the accounting system is necessary, the costs involved with integrating these systems may be much more than costs involved with maintenance of redundant databases. On the other hand, maintenance of redundant data increases the difficulty associated with ensuring that the data is correct, or updated in a timely manner, and ensuring that processes such as budget projections designed to manipulate the data, do so accurately, especially when databases are revised or differ between various engineering groups or disciplines.
One solution to combat the complexities involved with integration of loosely coupled, specialized systems and the maintenance of redundant databases and processes, is to incorporate a common interface such as a middleware application on each of the specialized systems. The middleware applications, such as IBM's message queuing MQSeries™ and WebSphere™ applications, facilitate messaging systems that allow one system to gather data from another system.
As the messaging systems between the multiple, specialized systems become more prevalent and systems become more dependent upon the communications, system failures become a more critical problem, causing other, linked systems to hang-up or fail. In particular, messages may be lost as a result of a system failure and, for some critical communications, the loss of a message can cause the process that requested data to fail. This problem is exacerbated when an intermediate system is involved with the communications. For example, a workstation may request data associated with a server. The server may generate the data by requesting and processing data from one or more hosts. If the server fails, the original message from the workstation and/or the requests for the hosts may be lost. When the server is restarted, the server may be unable to recover the message and associate the replies from the hosts with the message even though the workstation may still be waiting for the data requested by the original message.
Message persistence is implemented by such systems to attenuate the problems encountered as a result of system failures. Message persistence, or persisting the message, refers to storing the message in a manner that allows the message to be recovered after volatile memory is lost. For instance, a message queue may reside in volatile memory and, as a result of a system failure, the contents of the queue are lost or invalidated. A persisted message for that queue, which is stored in, e.g., non-volatile memory, can then be recovered, allowing the message to be processed. On the other hand, message persistence typically involves a trade-off with response time. In current solutions, upperware applications invoked to process messages may store the message to non-volatile data storage. Messages may be placed in an inbound queue upon acceptance and stored directly to the non-volatile storage after removing the message from the inbound queue. If a system failure occurs while processing the message, the original message can be found in the non-volatile memory and processing can start over. However, persisting the message to non-volatile memory via upperware can increase latencies in response time significantly and leave gaps in the persistence that may allow the message to be lost such as the gap between removing the message from the inbound queue and storing the message in non-volatile memory.